Trace-class Gaussian priors for Bayesian learning of neural networks with MCMC

Abstract This paper introduces a new neural network based prior for real valued functions. Each weight and bias of the has an independent Gaussian prior, with key novelty that variances decrease in width such way resulting function is well defined limit infinite network. We show induced posterior over functions amenable to Monte Carlo sampling using Hilbert space Markov chain (MCMC) methods. type MCMC stable under mesh refinement, i.e. acceptance probability does not degenerate as more parameters function's are introduced, even ad infinitum. demonstrate these advantages other priors, example Bayesian Reinforcement Learning.